{
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {},
  "cells": [
    {
      "metadata": {},
      "source": [
        "<td>\n",
        "   <a target=\"_blank\" href=\"https://labelbox.com\" ><img src=\"https://labelbox.com/blog/content/images/2021/02/logo-v4.svg\" width=256/></a>\n",
        "</td>"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "\n",
        "<td>\n",
        "\n",
        "<a href=\"https://colab.research.google.com/github/Labelbox/labelbox-python/blob/master/examples/basics/Export_V1_to_V2_migration_support.ipynb\" target=\"_blank\"><img\n",
        "src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"></a>\n",
        "</td>\n",
        "\n",
        "<td>\n",
        "\n",
        "<a href=\"https://github.com/Labelbox/labelbox-python/tree/master/examples/basics/Export_V1_to_V2_migration_support.ipynb\" target=\"_blank\"><img\n",
        "src=\"https://img.shields.io/badge/GitHub-100000?logo=github&logoColor=white\" alt=\"GitHub\"></a>\n",
        "</td>"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Export V1 to V2 migration "
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "**Export V1 will no longer be available in any version of the SDK starting on April 2024**. We recommend users to plan accordingly. \n",
        "\n",
        "This notebook is designed to help users identify alternative V2 export methods that can serve as replacements for V1 methods."
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "### Key changes included in export V2 methods ( ``export()`` and ``export_v2()``):\n",
        "1. Added flexibility to only export the data that is needed. The new methods include parameters and filters to give you more granular control over your exports.\n",
        "2. Added functionality to stream an **unlimited** number of data rows using ``export()`` (available on SDK >=3.56). Upgrading to `export()` is recommended as it is a more scalable solution.\n",
        "\n",
        "For complete details on how to use export V2 methods please see the [Export V2 methods](https://docs.labelbox.com/reference/label-export#export-v2-methods) documentation."
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "###  Export V1 deprecated methods: \n",
        "Project methods : \n",
        "1. ```project.export_labels()```\n",
        "2. ```project.label_generator()```\n",
        "3. ```project.export_queued_data_rows()```\n",
        "\n",
        "Dataset methods: \n",
        "1. ```dataset.export_data_rows()```\n",
        "\n",
        "Batch methods: \n",
        "1. ```batch.export_data_rows()```\n",
        "\n",
        "Model methods :\n",
        "1. ```model_run.export_labels()```\n",
        "\n"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Imports"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "!pip install -q \"labelbox[data]\""
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "import labelbox as lb\n",
        "import pprint\n",
        "pp = pprint.PrettyPrinter(width=30, compact=True)\n"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## API Key and Client\n",
        "See the developer guide for [creating an API key](https://docs.labelbox.com/reference/create-api-key)."
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "API_KEY = \"\"\n",
        "client = lb.Client(api_key=API_KEY)\n",
        "client.enable_experimental = True ## This is required if using the export() streamable method"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "# Exports V1 to V2 guidelines\n",
        "The follow sections will demonstrate how to use the export V2 methods to fetch data from your projects, datasets, batches and model runs. "
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "## Export labels from a project\n"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "PROJECT_ID = \"\"\n",
        "project = client.get_project(PROJECT_ID)"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "##### Export V1 (deprecated)  \n",
        "1. ```project.export_labels()``` \n",
        "    - Parameters:  \n",
        "        - ```download: bool = False```\n",
        "        - ```timeout_seconds: int = 1800```\n",
        "    - Output : (str | List[Dict[Any, Any]] | None)"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "For a comprehensive example of Export V1 ``project.export_labels()`` output, please refer to our documentation: [Export V1 sample export](https://docs.labelbox.com/reference/export-image-annotations#sample-project-export)"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "2. ```project.label_generator()```\n",
        "    - Parameters: \n",
        "        - ```timeout_seconds: int = 600```\n",
        "    - Output: LabelGenerator"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Single entry from the output of project.label_generator() (deprecated)\n",
        "# Label objects will not be deprecated.\n",
        "single_output_from_generator = \"\"\"\n",
        "\n",
        "Label(\n",
        "    uid='clrf5csho2ihx07ilffgp2fzj',\n",
        "    data=ImageData(\n",
        "        im_bytes=None,\n",
        "        file_path=None,\n",
        "        url='https://storage.googleapis.com/labelbox-datasets/image_sample_data/2560px-Kitano_Street_Kobe01s5s4110.jpeg',\n",
        "        arr=None\n",
        "    ),\n",
        "    annotations=[\n",
        "        ObjectAnnotation(\n",
        "            confidence=None,\n",
        "            name='bounding_box',\n",
        "            feature_schema_id='clrf5ck4a0b9b071paa9ncu15',\n",
        "            extra={\n",
        "                'instanceURI': 'https://api.labelbox.com/masks/feature/clrf5csvi6ofm07lsf9pygwvi?token=<token>'\n",
        "                'color': '#ff0000',\n",
        "                'feature_id': 'clrf5csvi6ofm07lsf9pygwvi',\n",
        "                'value': 'bounding_box',\n",
        "                'page': None,\n",
        "                'unit': None\n",
        "            },\n",
        "            value=Rectangle(\n",
        "                extra={},\n",
        "                start=Point(extra={}, x=2096.0, y=1264.0),\n",
        "                end=Point(extra={}, x=2240.0, y=1689.0)\n",
        "            ),\n",
        "            classifications=[]\n",
        "        ),\n",
        "        # Add more annotations as needed\n",
        "        # ...\n",
        "    ],\n",
        "    extra={\n",
        "        'Created By': 'aovalle@labelbox.com',\n",
        "        'Project Name': 'Image Annotation Import Demo',\n",
        "        'Created At': '2024-01-15T16:35:59.000Z',\n",
        "        'Updated At': '2024-01-15T16:51:56.000Z',\n",
        "        'Seconds to Label': 66.0,\n",
        "        'Agreement': -1.0,\n",
        "        'Benchmark Agreement': -1.0,\n",
        "        'Benchmark ID': None,\n",
        "        'Dataset Name': 'image-demo-dataset',\n",
        "        'Reviews': [],\n",
        "        'View Label': 'https://editor.labelbox.com?project=clrf5ckex09m9070x1te223u5&label=clrf5csho2ihx07ilffgp2fzj',\n",
        "        'Has Open Issues': 0.0,\n",
        "        'Skipped': False,\n",
        "        'media_type': 'image',\n",
        "        'Data Split': None,\n",
        "        'Global Key': '2560px-Kitano_Street_Kobe01s5s41102.jpeg'\n",
        "    }\n",
        ")\n",
        "\n",
        "\"\"\""
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "##### Export V2 \n",
        "\n",
        "For complete details on the supported filters abd parameters, including how they are used and what information is included, please see the [Export overview](https://docs.labelbox.com/reference/label-export#optional-parameters-and-filters) documentation.\n",
        "\n",
        "1. ```project.export()``` : Starting from SDK version 3.56, a streamable method is available, this method allows you to stream unlimited number of data rows. However, if you are using an earlier version, you can still utilize the ```export_v2()``` function with identical parameters. It's important to note that the output task type differs, and streaming data methods are not included in `export_v2()`.\n",
        "\n",
        "    - Parameters:  \n",
        "        - ```\"label_details\": True```\n",
        "        - ```\"attachments\": True```\n",
        "        - ```\"data_row_details\": True```\n",
        "        - ```\"project_details\": True```\n",
        "        - ```\"label_details\": True```\n",
        "        - ```\"performance_details\": True```\n",
        "    - Output: \n",
        "       - ```ExportTask```\n",
        "            - `ExportTask.has_result()` return type:  bool \n",
        "            - `ExportTask.has_errors()` return type: bool\n",
        "            - `ExportTask.get_stream()` return type: Stream[JsonConverterOutput]"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "## Set the export parameters to only export labels\n",
        "export_params= {\n",
        "  \"attachments\": True,\n",
        "  \"metadata_fields\": True,\n",
        "  \"data_row_details\": True,\n",
        "  \"project_details\": True,\n",
        "  \"label_details\": True,\n",
        "  \"performance_details\": True\n",
        "}\n",
        "# You also have the option to include additional filtering to narrow down the list of labels\n",
        "filters = {}\n",
        "\n",
        "export_task = project.export(params=export_params, filters=filters)\n",
        "export_task.wait_till_done()"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "# Provide results with JSON converter\n",
        "# Returns streamed JSON output strings from export task results/errors, one by one\n",
        "\n",
        "# Callback used for JSON Converter\n",
        "def json_stream_handler(output: lb.JsonConverterOutput):\n",
        "  print(output.json_str)\n",
        "\n",
        "if export_task.has_errors():\n",
        "  export_task.get_stream(\n",
        "  converter=lb.JsonConverter(),\n",
        "  stream_type=lb.StreamType.ERRORS\n",
        "  ).start(stream_handler=lambda error: print(error))\n",
        "\n",
        "if export_task.has_result():\n",
        "  export_json = export_task.get_stream(\n",
        "    converter=lb.JsonConverter(),\n",
        "    stream_type=lb.StreamType.RESULT\n",
        "  ).start(stream_handler=json_stream_handler)\n",
        "\n",
        "print(\"file size: \", export_task.get_total_file_size(stream_type=lb.StreamType.RESULT))\n",
        "print(\"line count: \", export_task.get_total_lines(stream_type=lb.StreamType.RESULT))\n"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## Export queued (\"To Label\") data rows from a project"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "##### Export V1 (deprecated):  \n",
        "1. ``project.export_queued_data_rows()`` :\n",
        "    - Parameters: \n",
        "        - ``timeout_seconds: int = 120``\n",
        "        - ``include_metadata: bool = False``\n",
        "    - Output: List[Dict[str, str]]"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Single entry from the output of project.export_queued_data_rows() (deprecated)\n",
        "single_output_example = \"\"\"\n",
        "[\n",
        "  {'id': 'clpouak6nap2g0783ajd1d6pf',\n",
        " 'createdAt': '2023-12-03T02:04:34.062Z',\n",
        " 'updatedAt': '2023-12-03T02:05:33.797Z',\n",
        " 'externalId': None,\n",
        " 'globalKey': 'b57c9ab2-304f-4c17-ba5f-c536f39a6a46',\n",
        " 'metadataFields': [],\n",
        " 'customMetadata': [],\n",
        " 'rowData': 'https://storage.googleapis.com/labelbox-developer-testing-assets/image/data_files/santa.jpeg',\n",
        " 'mediaAttributes': {'assetType': 'image',\n",
        "  'contentLength': 305973,\n",
        "  'height': 1333,\n",
        "  'mimeType': 'image/jpeg',\n",
        "  'subType': 'jpeg',\n",
        "  'superType': 'image',\n",
        "  'width': 2000}}\n",
        "]\n",
        "\n",
        "\"\"\""
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "##### Export V2\n",
        "For complete details on the supported filters and parameters, including how they are used and what information is included, please see the [Export overview](https://docs.labelbox.com/reference/label-export#optional-parameters-and-filters) documentation.\n",
        "\n",
        "1. ```project.export()``` : Starting from SDK version 3.56, a streamable method is available, this method allows you to stream unlimited number of data rows. However, if you are using an earlier version, you can still utilize the ```export_v2()``` function with identical parameters. It's important to note that the output task type differs, and streaming data methods are not included in `export_v2()`.\n",
        "\n",
        "    - Parameters (Minimum required parameters):  \n",
        "      - ```\"data_row_details\": True```\n",
        "      - ```\"project_details\": True```\n",
        "    - Required filters: \n",
        "      - ``` \"workflow_status\": \"ToLabel\"```\n",
        "    - Output: \n",
        "       - ```ExportTask```\n",
        "            - `ExportTask.has_result()` return type:  bool \n",
        "            - `ExportTask.has_errors()` return type: bool\n",
        "            - `ExportTask.get_stream()` return type: Stream[JsonConverterOutput]"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "export_params= {\n",
        " \"attachments\": True, # Set to true if you want to export attachments\n",
        "  \"metadata_fields\": True, # Set to true if you want to export metadata\n",
        "  \"data_row_details\": True,\n",
        "  \"project_details\": True\n",
        "}\n",
        "filters = {\n",
        "  \"workflow_status\": \"ToLabel\" ## Using this filter will only export queued data rows\n",
        "}\n",
        "\n",
        "# An ExportTask is returned, this provides additional information about the status of your task, such as\n",
        "# any errors encountered and includes additional methods to stream your data\n",
        "\n",
        "\n",
        "export_task = project.export(params=export_params, filters=filters)\n",
        "export_task.wait_till_done()\n"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "# Provide results with JSON converter\n",
        "# Returns streamed JSON output strings from export task results/errors, one by one\n",
        "\n",
        "# Callback used for JSON Converter\n",
        "def json_stream_handler(output: lb.JsonConverterOutput):\n",
        "  print(output.json_str)\n",
        "\n",
        "\n",
        "if export_task.has_errors():\n",
        "  export_task.get_stream(\n",
        "\n",
        "  converter=lb.JsonConverter(),\n",
        "  stream_type=lb.StreamType.ERRORS\n",
        "  ).start(stream_handler=lambda error: print(error))\n",
        "\n",
        "if export_task.has_result():\n",
        "  export_json = export_task.get_stream(\n",
        "    converter=lb.JsonConverter(),\n",
        "    stream_type=lb.StreamType.RESULT\n",
        "  ).start(stream_handler=json_stream_handler)\n",
        "\n",
        "print(\"file size: \", export_task.get_total_file_size(stream_type=lb.StreamType.RESULT))\n",
        "print(\"line count: \", export_task.get_total_lines(stream_type=lb.StreamType.RESULT))"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## Export data rows from a Dataset "
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "DATASET_ID = \"\"\n",
        "dataset = client.get_dataset(DATASET_ID)"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "#### Export V1 (deprecated): \n",
        "\n",
        "1. ```dataset.export_data_rows()```\n",
        "  - Parameters:  \n",
        "    - ``timeout_seconds=120``\n",
        "    - ``include_metadata: bool = True``\n",
        "  - Output: \n",
        "    - Data row object generator\n"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Single entry from the output of dataset.export_data_rows() (deprecated)\n",
        "# Data row objects will not be deprecated.\n",
        "\n",
        "single_output_from_data_row_generator = \"\"\"\n",
        "<DataRow {\n",
        "    \"created_at\": \"2023-12-03 02:04:34.062000+00:00\",\n",
        "    \"external_id\": null,\n",
        "    \"global_key\": \"b57c9ab2-304f-4c17-ba5f-c536f39a6a46\",\n",
        "    \"media_attributes\": {\n",
        "        \"assetType\": \"image\",\n",
        "        \"contentLength\": 305973,\n",
        "        \"height\": 1333,\n",
        "        \"mimeType\": \"image/jpeg\",\n",
        "        \"subType\": \"jpeg\",\n",
        "        \"superType\": \"image\",\n",
        "        \"width\": 2000\n",
        "    },\n",
        "    \"metadata\": [],\n",
        "    \"metadata_fields\": [],\n",
        "    \"row_data\": \"https://storage.googleapis.com/labelbox-developer-testing-assets/image/data_files/santa.jpeg\",\n",
        "    \"uid\": \"clpouak6nap2g0783ajd1d6pf\",\n",
        "    \"updated_at\": \"2023-12-03 02:05:33.797000+00:00\"\n",
        "}>\n",
        "\"\"\""
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "#### Export V2\n",
        "For complete details on the supported filters and parameters, including how they are used and what information is included, please see the [Export overview](https://docs.labelbox.com/reference/label-export#optional-parameters-and-filters) documentation.\n",
        "\n",
        "1. ```project.export()``` : Starting from SDK version 3.56, a streamable method is available, this method allows you to stream unlimited number of data rows. However, if you are using an earlier version, you can still utilize the ```export_v2()``` function with identical parameters. It's important to note that the output task type differs, and streaming data methods are not included in `export_v2()`.\n",
        "\n",
        "    - Parameters (minimum required parameters):  \n",
        "      - ``\"data_row_details\": True``\n",
        "    - Output: \n",
        "       - ```ExportTask```\n",
        "            - `ExportTask.has_result()` return type:  bool \n",
        "            - `ExportTask.has_errors()` return type: bool\n",
        "            - `ExportTask.get_stream()` return type: Stream[JsonConverterOutput]"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "export_params = {\n",
        " \"attachments\": True, # Set to true if you want to export attachments\n",
        "  \"metadata_fields\": True, # Set to true if you want to export metadata\n",
        "  \"data_row_details\": True,\n",
        "}\n",
        "filters = {}\n",
        "\n",
        "# A task is returned, this provides additional information about the status of your task, such as\n",
        "# any errors encountered\n",
        "export_task = dataset.export(params=export_params, filters=filters)\n",
        "export_task.wait_till_done()"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "# Provide results with JSON converter\n",
        "# Returns streamed JSON output strings from export task results/errors, one by one\n",
        "\n",
        "# Callback used for JSON Converter\n",
        "def json_stream_handler(output: lb.JsonConverterOutput):\n",
        "  print(output.json_str)\n",
        "\n",
        "\n",
        "if export_task.has_errors():\n",
        "  export_task.get_stream(\n",
        "\n",
        "  converter=lb.JsonConverter(),\n",
        "  stream_type=lb.StreamType.ERRORS\n",
        "  ).start(stream_handler=lambda error: print(error))\n",
        "\n",
        "if export_task.has_result():\n",
        "  export_json = export_task.get_stream(\n",
        "    converter=lb.JsonConverter(),\n",
        "    stream_type=lb.StreamType.RESULT\n",
        "  ).start(stream_handler=json_stream_handler)\n",
        "\n",
        "print(\"file size: \", export_task.get_total_file_size(stream_type=lb.StreamType.RESULT))\n",
        "print(\"line count: \", export_task.get_total_lines(stream_type=lb.StreamType.RESULT))"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## Export data rows from a batch"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "#### Export V1 (deprecated): \n",
        "1. ```batch.export_data_rows()```\n",
        "  - Parameters:  \n",
        "    - ``timeout_seconds=120``\n",
        "    - ``include_metadata: bool = True``\n",
        "  - Output: \n",
        "    - Data row object generator"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Single output from batch.export_data_rows() method (deprecated)\n",
        "# Data row objects will not be deprecated\n",
        "\n",
        "single_output_from_data_row_generator = \"\"\"\n",
        "<DataRow {\n",
        "    \"created_at\": \"2023-12-03 02:04:34.062000+00:00\",\n",
        "    \"external_id\": null,\n",
        "    \"global_key\": \"b57c9ab2-304f-4c17-ba5f-c536f39a6a46\",\n",
        "    \"media_attributes\": {\n",
        "        \"assetType\": \"image\",\n",
        "        \"contentLength\": 305973,\n",
        "        \"height\": 1333,\n",
        "        \"mimeType\": \"image/jpeg\",\n",
        "        \"subType\": \"jpeg\",\n",
        "        \"superType\": \"image\",\n",
        "        \"width\": 2000\n",
        "    },\n",
        "    \"metadata\": [],\n",
        "    \"metadata_fields\": [],\n",
        "    \"row_data\": \"https://storage.googleapis.com/labelbox-developer-testing-assets/image/data_files/santa.jpeg\",\n",
        "    \"uid\": \"clpouak6nap2g0783ajd1d6pf\",\n",
        "    \"updated_at\": \"2023-12-03 02:05:33.797000+00:00\"\n",
        "}>\n",
        "\"\"\""
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "#### Export V2\n",
        "For complete details on the supported filters and parameters, including how they are used and what information is included, please see the [Export overview](https://docs.labelbox.com/reference/label-export#optional-parameters-and-filters) documentation.\n",
        "\n",
        "1. ```project.export()``` : Starting from SDK version 3.56, a streamable method is available, this method allows you to stream unlimited number of data rows. However, if you are using an earlier version, you can still utilize the ```export_v2()``` function with identical parameters. It's important to note that the output task type differs, and streaming data methods are not included in `export_v2()`.\n",
        "\n",
        "    - Required parameters:  \n",
        "      - ```\"data_row_details\": True```,\n",
        "      - ```\"batch_ids\": [<batch_id>] ```\n",
        "    - Output: \n",
        "       - ```ExportTask```\n",
        "            - `ExportTask.has_result()` return type:  bool \n",
        "            - `ExportTask.has_errors()` return type: bool\n",
        "            - `ExportTask.get_stream()` return type: Stream[JsonConverterOutput]"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Find the batch ID by navigating to \"Batches\" -->  \"Manage batches\" --> \"Copy Batch ID\"\n",
        "BATCH_ID = \"\""
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "export_params = {\n",
        " \"attachments\": True,\n",
        "  \"metadata_fields\": True,\n",
        "  \"data_row_details\": True,\n",
        "  \"project_details\": True,\n",
        "  \"performance_details\": True,\n",
        "  \"batch_ids\" : [BATCH_ID] # Include batch ids if you only want to export specific batches, otherwise,\n",
        "  #you can export all the data without using this parameter\n",
        "}\n",
        "filters = {}\n",
        "\n",
        "# A task is returned, this provides additional information about the status of your task, such as\n",
        "# any errors encountered\n",
        "export_task = project.export(params=export_params, filters=filters)\n",
        "export_task.wait_till_done()"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "# Provide results with JSON converter\n",
        "# Returns streamed JSON output strings from export task results/errors, one by one\n",
        "\n",
        "# Callback used for JSON Converter\n",
        "def json_stream_handler(output: lb.JsonConverterOutput):\n",
        "  print(output.json_str)\n",
        "\n",
        "\n",
        "if export_task.has_errors():\n",
        "  export_task.get_stream(\n",
        "\n",
        "  converter=lb.JsonConverter(),\n",
        "  stream_type=lb.StreamType.ERRORS\n",
        "  ).start(stream_handler=lambda error: print(error))\n",
        "\n",
        "if export_task.has_result():\n",
        "  export_json = export_task.get_stream(\n",
        "    converter=lb.JsonConverter(),\n",
        "    stream_type=lb.StreamType.RESULT\n",
        "  ).start(stream_handler=json_stream_handler)\n",
        "\n",
        "print(\"file size: \", export_task.get_total_file_size(stream_type=lb.StreamType.RESULT))\n",
        "print(\"line count: \", export_task.get_total_lines(stream_type=lb.StreamType.RESULT))"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## Export data rows from a Model "
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "#### Export V1 (deprecated): \n",
        "1. ```model_run.export_labels(downlaod=True)```\n",
        "    - Parameters:  \n",
        "        - ```download: bool = False```\n",
        "        - ```timeout_seconds: int = 1800```\n",
        "    - Output : (str | List[Dict[Any, Any]] | None)"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "# Single output from model_run.export_labels()\n",
        "single_output_example = \"\"\"\n",
        "[\n",
        "   {'ID': '1c48a7a0-3016-48e0-b0e3-47430f974869',\n",
        "   'Data Split': 'training',\n",
        "   'DataRow ID': 'clpqdyf650xd40712pycshy6a',\n",
        "   'External ID': './resume/BANKING/99124477.pdf',\n",
        "   'Labeled Data': 'https://storage.labelbox.com/cl5bn8qvq1av907xtb3bp8q60%2F8c6afc38-42a4-b2e1-a2e3-1e3b0c2998fc-99124477.pdf?Expires=1706637969726&KeyName=labelbox-assets-key-3&Signature=2nVt3sJ21CbjGS9I64yFquUELRw',\n",
        "   'Media Attributes': {'assetType': 'pdf',\n",
        "      'contentLength': 42535,\n",
        "      'mimeType': 'application/pdf',\n",
        "      'pageCount': 3,\n",
        "      'subType': 'pdf',\n",
        "      'superType': 'application'},\n",
        "   'Label': {'objects': [{'featureId': 'b9f3b584-0f45-050a-88d4-39c2a169c8e1',\n",
        "      'schemaId': 'clq1ckwbd08jp07z91q9mch5j',\n",
        "      'title': 'Test',\n",
        "      'value': 'test',\n",
        "      'color': '#1CE6FF',\n",
        "      'data': {'location': [{'text-bbox': {'page': 1,\n",
        "            'top': 158.44,\n",
        "            'left': 58.765,\n",
        "            'height': 13.691,\n",
        "            'width': 78.261}}],\n",
        "         'unit': 'POINTS'}}],\n",
        "      'classifications': [],\n",
        "      'relationships': []}}\n",
        "   ]\n",
        "   \"\"\"\n"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "#### Export V2\n",
        "For complete details on the supported filters and parameters, including how they are used and what information is included, please see the [Export overview](https://docs.labelbox.com/reference/label-export#optional-parameters-and-filters) documentation.\n",
        "\n",
        "1. ```model_run.export()```: Starting from SDK version 3.56, a streamable method is available, this method allows you to stream unlimited number of data rows. However, if you are using an earlier version, you can still utilize the ```export_v2()``` function with identical parameters. It's important to note that the output task type differs, and streaming data methods are not included in `export_v2()`.\n",
        "\n",
        "    - Required parameters:  \n",
        "      - ```\"data_row_details\": True```\n",
        "      - ```\"project_details\": True```\n",
        "      - ```\"label_details\": True```\n",
        "    - Required filters: \n",
        "      - N/A -> Filters not supported\n",
        "    - Output: \n",
        "       - ```ExportTask```\n",
        "            - `ExportTask.has_result()` return type:  bool \n",
        "            - `ExportTask.has_errors()` return type: bool\n",
        "            - `ExportTask.get_stream()` return type: Stream[JsonConverterOutput]"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "MODEL_RUN_ID = \"\"\n",
        "model_run = client.get_model_run(MODEL_RUN_ID)"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "export_params = {\n",
        " \"attachments\": True,\n",
        "  \"metadata_fields\": True,\n",
        "  \"data_row_details\": True,\n",
        "  \"project_details\": True,\n",
        "  \"performance_details\": True\n",
        "}\n",
        "\n",
        "export_task = model_run.export(params=export_params)\n",
        "export_task.wait_till_done()\n"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "# Provide results with JSON converter\n",
        "# Returns streamed JSON output strings from export task results/errors, one by one\n",
        "\n",
        "# Callback used for JSON Converter\n",
        "def json_stream_handler(output: lb.JsonConverterOutput):\n",
        "  print(output.json_str)\n",
        "\n",
        "if export_task.has_errors():\n",
        "  export_task.get_stream(\n",
        "  converter=lb.JsonConverter(),\n",
        "  stream_type=lb.StreamType.ERRORS\n",
        "  ).start(stream_handler=lambda error: print(error))\n",
        "\n",
        "if export_task.has_result():\n",
        "  export_json = export_task.get_stream(\n",
        "    converter=lb.JsonConverter(),\n",
        "    stream_type=lb.StreamType.RESULT\n",
        "  ).start(stream_handler=json_stream_handler)\n",
        "\n",
        "print(\"file size: \", export_task.get_total_file_size(stream_type=lb.StreamType.RESULT))\n",
        "print(\"line count: \", export_task.get_total_lines(stream_type=lb.StreamType.RESULT))"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "## Export data rows from a video project\n",
        "Video projects include additional fields. Please refer to the example below to extract specific fields from video exports.\n"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "##### Export V1 (deprecated)  \n",
        "1. ```project.export_labels()``` \n",
        "    - Parameters:  \n",
        "        - ```download: bool = False```\n",
        "        - ```timeout_seconds: int = 1800```\n",
        "    - Output : (str | List[Dict[Any, Any]] | None)"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "For a comprehensive example of Export V1 ``project.export_labels()`` output, please refer to our documentation: [Export V1 sample export](https://docs.labelbox.com/reference/export-image-annotations#sample-project-export)"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "2. ```project.video_label_generator()```\n",
        "    - Parameters: \n",
        "        - ```timeout_seconds: int = 600```\n",
        "    - Output: LabelGenerator"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "##### Export V2\n",
        "\n",
        "For complete details on the supported filters and parameters, including how they are used and what information is included, please see the [Export overview](https://docs.labelbox.com/reference/label-export#optional-parameters-and-filters) documentation.\n",
        "\n",
        "1. ```project.export()```: Starting from SDK version 3.56, a streamable method is available, this method allows you to stream unlimited number of data rows. However, if you are using an earlier version, you can still utilize the ```export_v2()``` function with identical parameters. It's important to note that the output task type differs, and streaming data methods are not included in `export_v2()`.\n",
        "\n",
        "    - Required parameters:  \n",
        "      - ```\"attachments\": True```\n",
        "      - ```\"data_row_details\": True```\n",
        "      - ```\"project_details\": True```\n",
        "      - ```\"label_details\": True```\n",
        "      - ```\"performance_details\": True```\n",
        "    - Output: \n",
        "       - ```ExportTask```\n",
        "            - `ExportTask.has_result()` return type:  bool \n",
        "            - `ExportTask.has_errors()` return type: bool\n",
        "            - `ExportTask.get_stream()` return type: Stream[JsonConverterOutput]"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "VIDEO_PROJECT_ID = \"\"\n",
        "project = client.get_project(VIDEO_PROJECT_ID)"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "export_params = {\n",
        " \"attachments\": True,\n",
        "  \"metadata_fields\": True,\n",
        "  \"data_row_details\": True,\n",
        "  \"project_details\": True,\n",
        "  \"performance_details\": True,\n",
        "  \"label_details\": True,\n",
        "  \"interpolated_frames\": True # For additional information on interpolated frames please visit our documentation https://docs.labelbox.com/docs/video-annotations#video-editor-components\n",
        "}\n",
        "filters = {}\n",
        "\n",
        "# A task is returned, this provides additional information about the status of your task, such as\n",
        "# any errors encountered\n",
        "export_task = project.export(params=export_params, filters=filters)\n",
        "export_task.wait_till_done()"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "Fetch frame specific objects and frame or global classifications"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "import json\n",
        "import pprint as pp  # Assuming pp is imported from pprint module\n",
        "\n",
        "frames_objects_class_list = []\n",
        "global_class_list = []\n",
        "\n",
        "stream = export_task.get_stream()\n",
        "for output in stream:\n",
        "    output_json = json.loads(output.json_str)\n",
        "    for dr in output_json[\"projects\"][VIDEO_PROJECT_ID][\"labels\"]:\n",
        "        frames_data = dr[\"annotations\"][\"frames\"]\n",
        "        for k, v in frames_data.items():\n",
        "            frames_objects_class_list.append({k: v})\n",
        "        global_class_list.extend(dr[\"annotations\"][\"classifications\"])\n",
        "\n",
        "    print(\"------- Frame specific classifications and objects -------\")\n",
        "    pp.pprint(frames_objects_class_list)\n",
        "\n",
        "    print(\"------ Global classifications -------\")\n",
        "    pp.pprint(global_class_list)\n",
        "\n",
        "\n"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "Fetch key frame feature map "
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "keyframe_map = []\n",
        "\n",
        "stream = export_task.get_stream()\n",
        "for output in stream:\n",
        "    output_json = json.loads(output.json_str)\n",
        "    labels = output_json[\"projects\"][VIDEO_PROJECT_ID][\"labels\"]\n",
        "    for label in labels:\n",
        "        annotations = label[\"annotations\"][\"key_frame_feature_map\"]\n",
        "        for key, value in annotations.items():\n",
        "            keyframe_map.append({key: value})\n",
        "\n",
        "print(\"----- Keyframe Feature Map -----\")\n",
        "pp.pprint(keyframe_map)"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    },
    {
      "metadata": {},
      "source": [
        "Fetch segments "
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {},
      "source": [
        "segments_map = []\n",
        "stream = export_task.get_stream()\n",
        "for output in stream:\n",
        "    output_json = json.loads(output.json_str)\n",
        "    labels = output_json[\"projects\"][VIDEO_PROJECT_ID][\"labels\"]\n",
        "    for label in labels:\n",
        "        annotations = label[\"annotations\"][\"segments\"]\n",
        "        for key, value in annotations.items():\n",
        "            segments_map.append({key: value})\n",
        "\n",
        "print(\"----- Segments Feature Map -----\")\n",
        "pp.pprint(segments_map)"
      ],
      "cell_type": "code",
      "outputs": [],
      "execution_count": null
    }
  ]
}